Systems and methods for generating interactable elements in text strings relating to media assets

ABSTRACT

Systems and methods for improving displays of media assets are disclosed herein. In an embodiment, a system receives a plurality of text comments from a plurality of devices to which a media asset was transmitted. The system analyzes the comments to identify text strings within the text comments. The system generates interactable elements from the text strings in the text comments, such that an interaction with the text string causes display of identifiers of media assets corresponding to the text string.

BACKGROUND

This disclosure is generally directed to graphical user interfaces fordisplaying media assets. In particular, methods and systems are providedfor modifying text strings in text comments displayed on graphical userinterfaces to include interactable elements.

SUMMARY

Modernly, media assets, such as videos, music, podcasts, or images areprovided to users through a graphical user interface. The graphical userinterface may additionally include recommendations of other videos orother supplemental content. Unfortunately, identifying recommendationscan be extremely difficult and the display of bad recommendations canclutter the interface or require additional searches which increases thecomputational load on the media server. In some approaches,recommendations are provided for any other available media. Theseapproaches have the same effect as generating and displaying norecommendations, as both require additional searching without displayingthe best recommendations in an easily identifiable location.

To address the aforementioned problem, in one approach, recommendationsare provided based on metadata of the media asset, metadata of theviewing device, and/or popularity of other media assets. For instance, adisplayed media asset may have metadata including a title and one ormore “tags” which comprise keywords identifying terms relating to themedia asset. Recommendations may be generated through a search of othermedia assets for media assets that match the tags of the video or thetitle of the video. Alternatively, recommendations may be generatedbased on user preferences which identify types of media assets orsources of media assets that are enjoyed by a user.

While the above approach does provide many options for generatingrecommendations, the recommendations do not take into account a way auser is interacting with a media asset. For instance, many graphicaluser interfaces through which media assets are displayed provide optionsthrough which viewing devices can provide comments on the media assets.The comments can include text comments, video comments, image comments,audio comments, or any other media provided by the user in relation tothe video. Device interactions with the media asset through comments mayindicate different video preferences. For instance, a video aboutcooking may include a comment relating to a special knife that was used.If the knife is unidentified in the tags or title of the video, thespecial knife will not be used to generate recommendations orsupplemental content.

To overcomes such deficiencies, methods and systems are described hereinfor leveraging information in comments received from a plurality ofdevices to which a media asset was displayed to modify the media asset,media asset recommendations, or the comment interface. The presentdisclosure addresses the problem of generating recommendations withincomplete information by analyzing comment data to generate newinformation through which recommendations can be generated. The presentdisclosure additionally provides additional methods for generatingsearch interfaces, thereby providing a less cluttered interface that isvisually navigable.

In some embodiments, text of comments on a media asset are analyzed forterms that do not match recommended media assets. The terms that do notmatch recommended assets are then analyzed to determine whether searchresults should be generated based on the terms. The analysis may takeinto account metadata of a device that posted the comment, a length ofthe term, a frequency of the term in the comments, previous searches forthe term, or whether the term matches a location, product, person, orother known entity.

In some embodiments, the comments are modified to include interactableelements, such as a hyperlink. For instance, if the media serverdetermines that the term “Japanese gardens” should be used to generatesearch results, the media server may modify the term “Japanese gardens”to include an interactable element such that search results for the term“Japanese gardens” are displayed in response to a selection of theinteractable element.

In some embodiments, the interactable elements, when selected causedisplaying search results based on the comment text in addition toprevious recommendations and the media asset, such as through an overlayof the graphical user interface. The additional display of the searchresults based on the comment provides an additional tool for searchingfor content through terms in text comments.

In some embodiments, the search results include connections to otherplatforms through an application programming interface or hyperlinks.The media server may utilize a datastore corresponding to the otherplatforms to determine matches between the term and the other platform,and provide a link to the other platform. Examples include retailpurchasing platforms, travel platforms, map platforms, or other mediaplatforms.

In some embodiments, the comments are analyzed to determine if text inthe comments relates to supplemental content. If text of the commentsrelates to supplemental content, the media server may modify the mediaasset to display the supplemental content. In some embodiments, themedia asset and the comment are analyzed to identify a location and/ortime in the media asset to display the supplemental content.

DESCRIPTION OF THE DRAWINGS

The above and other objects and advantages of the disclosure will beapparent upon consideration of the following detailed description, takenin conjunction with the accompanying drawings, in which like referencecharacters refer to like parts throughout, and in which:

FIG. 1 depicts an example system for generating interactable textstrings in comments based on text analysis.

FIG. 2 depicts an example embodiment of a user interface where text incomments is identified for generating a multimedia link.

FIG. 3 depicts an example embodiment of a user interface where text incomments is replaced with an interactable element.

FIG. 4 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes simultaneous display of amedia asset, identifiers of recommended media assets, and identifiers ofmedia assets identified based on the text of the multimedia link.

FIG. 5 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes display of identifiers ofmedia assets based on the text of the multimedia link to be displayed ina recommendations section of the user interface.

FIG. 6 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes display of identifiers ofmedia assets hosted by different platforms.

FIG. 7 depicts an example embodiment of a user interface where inputinteraction with a displayed identifier of a media asset hosted by adifferent platform causes display of an additional window correspondingto the different platform.

FIG. 8 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes display of identifiers ofmedia assets based on an identified location.

FIG. 9 depicts an example embodiment of modifying display of a mediaasset with supplemental content based on data from a text comment.

FIG. 10 depicts an example method for generating interactable textstrings in comments based on text analysis.

FIG. 11 depicts an example method for generating interactable textstrings in comments based on a determination that a comment containsadditional information.

FIG. 12 depicts an example method for dynamically routing notificationsbased on dynamically identified rules based on the received notification

FIG. 13 depicts a generalized embodiment of an illustrative device.

FIG. 14 depicts a block diagram of an illustrative media system, inaccordance with some embodiments of the disclosure.

DETAILED DESCRIPTION

FIG. 1 depicts an example system for generating interactable textstrings in comments based on text analysis. FIG. 1 includes a mediaserver 120, user devices 130, and a viewer device 140. FIG. 1 provides apractical example of a system; one skilled in the art would recognizethat more or less suitable elements may be used to perform the methodsdescribed herein. For example, a first server may transmit the mediaassets while a second server performs the comment analysis. As anotherexample, the media server 120 may communicate with an external server,such as through an application programming interface (API), to identifyoptions for displaying in response to a selection of an interactiveelement.

In the example embodiment of FIG. 1 , at step 102, a media server 120provides a media asset to user devices 130. The media server 120 mayprovide the media asset through an application hosted by the mediaserver 120 and/or through a separate application, such as a webpageaccessed through a browser. The media asset may comprise video, audio,images, and/or any combination thereof. For instance, the media assetmay be any of a stored video, a streamed video, an image, such as animage of a product for purchase or an image of a location for travel, anaudio recording, such as a podcast or song, or any other type of mediaasset, such as a graphics interchange format (GIF) image.

In the example embodiment of FIG. 1 , at step 104, the user devices 130transmit text comments for the media asset to the media server 120. Forexample, the media server 120 may provide a graphical user interfacethrough which text comments may be generated with respect to theprovided media asset. The graphical user interface may additionallyprovide other comment options, such as image, video, or audio. In someembodiments, media server 120 additionally provides suggestions for textin the comments based on metadata of the media asset and/or based ontext strings in previously submitted comments identified through themethods described herein. Through the graphical user interface,different user devices 130 may input different text comments. When a“submit” option is selected after text has been entered into thegraphical user interface, a text comment is transmitted to the mediaserver 120. The text comment may additionally be stored with metadataidentifying a profile corresponding to the comment submission, such asuser profile information and/or metadata relating to the video. The userprofile information may include demographic information, previous searchhistories, previous application usage, previous comments, or otherinformation relating to a user profile through which the comment wasgenerated.

In the example embodiment of FIG. 1 , at step 106, the media server 120analyzes the text in the text comments and generates multimedia linksfrom the analyzed text. For example, the media server 120 may analyzethe text comments using the methods described herein to identify a textstring in at least one of the text comments. The media server 120 usesthe text string to generate one or more results, such as through asearch of media assets of the media server 120 or a search forexternally provided media assets through other applications and/orserver computers. The media server 120 modifies the text comment togenerate a multimedia link from the text string. The multimedia link maytake the appearance of the text string in the comment, such that thetext string in the comment may be selectable.

In the example embodiment of FIG. 1 , at step 108, the media server 120transmits the media asset with the interactive text comments to a viewerdevice 140. For example, the media server 120 may provide a graphicaluser interface to viewer device 140 which includes the media asset andthe text comments received from user devices 130. The media server 120may cause display of the media asset and the text comments through thegraphical user interface, such that a viewer of the media asset has theoption to view the text comments, including a text comment modified withthe multimedia link. Practical examples for identifying text strings,modifying text strings, and displaying results when a multimedia link isselected are described further herein with respect to FIG. 2-9 .

FIG. 2 depicts an example embodiment of a user interface where text incomments is identified for generating a multimedia link. Each of FIG.2-9 provide examples of graphical user interfaces that are displayed onone or more computing devices for performing the methods describedherein. Other embodiments may utilize more or less interface elements.Additionally, why FIG. 2-9 depict an interface where the media assetprovided comprises a video, other embodiments may be performed where theprovided media asset comprises any other type of media asset, such asaudio, text, or images.

Interface 200 comprises a first media asset 210, first media assetinformation 212, identifiers 214 of a plurality of second media assets,and text comments 216.

The first media asset 210 comprises a media asset provided by a mediaserver, such as in response to a request from a user device to play themedia asset. In the example of FIG. 2 , the media asset is a video withthe title “Top Ten Video Games of All Time.” The interface may includeoptions to play the video while continuing to display the text comments216 and the identifiers 214. The first media asset information 212comprises information relating to the first media asset, such as a titleof the first media asset, a description of the first media asset, a userprofile through which the first media asset was posted, or tagscomprising terms related to the first media asset. The first media assetinformation 212 may comprise information generated through input from aclient computing device prior to or after uploading of the first mediaasset by the client computing device or other computing device that issigned into a same user account as the client computing device. Othermetadata relating to the first media asset 210 may be stored but notdisplayed in the first media asset information 212. For example, in someembodiments, tags may be generated for the video but not displayed inthe first media asset information 212.

Identifiers 214 comprise identifiers of a plurality of different mediaassets which are recommended for viewing. The plurality of differentmedia assets may be recommended based on popularity, informationrelating to a user profile of a viewer such as a past viewing history ordata defining subscribed channels, metadata of the first media asset, ora combination thereof. Metadata of the first media asset may include thetitle of the first media asset, keywords in the description of the firstmedia asset, a defined genre of the first media asset, and/or tagsgenerated for the first media asset. The media server may perform asearch through media assets provided by the media server using termsfrom the metadata of the first media asset.

Text comments 216 comprise a plurality of comments generated for thefirst media asset, such as through a computing device which has providedauthentication information for a user profile. Each of text comments 216comprises comment text 218, commentor information 220, and commentreactions 222. Comment text 218 comprises information previously enteredinto a comment field by one or more users. For example, the comment textfor the first comment reads “How is Fortnite on this list but notMinecraft?” The aforementioned comment text may have been entered into acomment field corresponding to the first media asset and submitted by acomputing device. The commentor information may include metadatacorresponding to the computing device, such as user profile information,submission time, commentor location, or other information relating tothe commentor or the comments. The user profile information may includea user profile image, description, and/or user name. Additional userprofile information may be stored with respect to the user account butnot displayed on the interface, such as descriptions, past videopostings, or other information relating to the profile or device.

Comment reactions comprise one or more indicators of responses to thetext comment received from other computing devices. For example, acomputing device may display a text comment with an option to leave areaction, such as to indicate approval, disapproval, or other emotion,or to leave a reply comment with additional text, images, audio, orvideo. The computing device may receive input selecting an option toleave a reaction and transmit data indicating the selection to the mediaserver which aggregates the reactions and displays them on theinterface. Thus, the first comment in FIG. 2 received a reactionselection from 576 different devices.

The text 218 of the text comments may be analyzed to identify one ormore text strings for turning into interactable elements. In theembodiment of FIG. 2 , three text strings were identified based on theanalysis, “Minecraft” in the first text comment, “Monokuma” in thesecond text comment, and “Akihabara” in the third text comment. WhileFIG. 2 depicts only single word text strings, embodiments describedherein may identify text strings comprising any number of words.Additionally, while in FIG. 2 a single text string is identified in eachcomment, embodiments described herein may identify multiple text stringsin a single comment and/or identify text strings in only a subset of thecomments.

Text strings in the text comments may be identified through analysis ofthe text comments using any of a number of techniques including, but notlimited to, identifying known keywords in the text comments, identifyingtext strings based on number of occurrences, identifying text stringsbased on uniqueness of terms, identifying descriptive words in textstrings, identifying comments with a highest number of interactions,identifying text strings repeated in replies to comments, identifyingpreviously searched terms, identifying matching products, identifyingmatching locations, or any combination thereof. Each of theaforementioned methods are described further herein.

Identifying known keywords in the text comments may comprise storingdata identifying key words. For example, metadata for a plurality ofmedia assets may include tags that are generated by the uploader of themedia asset, other viewers, or the media server which comprise keywordsfor identifying the media asset in the search. The media server mayaggregate the tags for the videos and store data identifying each of thetags as a keyword. In an embodiment, only a subset of the tags arestored as keywords, such as tags that have been repeated a thresholdnumber of times across media assets and/or a top percentage, such as topten percent, of tags based on a number of times the tags have beenrepeated across media assets. The media server may iterate through wordsor groupings of words and compare the identified words to the storedkeywords. Words that match the stored keywords may be identified ascandidates for generating interactable elements.

Identifying text strings based on number of occurrences may comprisecomparing text strings across the comments to identify a number ofoccurrences of the text strings. In an embodiment, the systemdetermines, for each text string, a number of times the text stringoccurred within the text comments for the media asset and a length ofthe text string. The media server may be configured to identify, ascandidates for generating interactable elements, text strings thatmaximize both a number of occurrences and a text string length. Forinstance, the media server may compute a candidate value for each textstring as:

C=w ₁ O+w ₂ l

where C is the candidate value, O is the number of occurrences, l is thetext string length, and w₁ and w₂ are weights that are preselected, suchas 0.5 for w₁ and 1 for w₂. The media server may select as candidates atop percentage of the text strings based on the candidate value or eachtext string with a candidate value greater than a threshold.

Identifying text strings based on uniqueness of terms may comprisecomparing text strings for the media asset to text strings in commentsor metadata of other media assets. For example, the media server maystore data identifying different text strings across a plurality ofmedia assets and a number of instances of those text strings. The mediaserver may identify text strings that are uncommon across other mediaassets, such as text strings with a number of instances less than athreshold number of in a bottom percentage, such as a lower ten percent,of number of instances. The media server may additionally determine anumber of occurrences of the text string in comments for the mediaasset. The media server may be configured to identify, as candidates forgenerating interactable elements, text strings that minimize a number ofoccurrences across the plurality of media assets while maximizing anumber of occurrences for the media asset. For instance, the mediaserver may compute a candidate value for each text string as:

$C = {{w_{1}O_{m}} - {w_{2}{\sum\limits_{j = 1}^{n}\frac{O_{j}}{n}}}}$

where O_(m) is the number of occurrences for the current media asset,O_(j) is the number of occurrences for the jth media asset of the nmedia assets, and w₁ and w₂ are weights that are preselected, such as0.5 for w₁ and 1 for w₂. The media server may select as candidates a toppercentage of the text strings based on the candidate value or each textstring with a candidate value greater than a threshold.

Identifying descriptive words in text strings may comprise storing dataidentifying descriptive words. For example, the media server may storedata identifying a plurality of common descriptors, such as countrydesignations (e.g. “Japanese”) or vehicle descriptors (e.g. “model”).The media server may iterate through words or groupings of words andcompare the identified words to the stored descriptors. Words that matchthe stored descriptors may be used to identify candidates for generatinginteractable elements. For example, the media server may identify a nounphrase that begins with the descriptor in the text and select the nounphase as a candidate syllable.

Identifying text strings based on comments with the highest number ofinteractions may comprise determining, for the text comments, a numberof interactions of one or more types. The types may include reading thecomment, hovering or positioning a mouse cursor over the comment,selecting an option to leave an interaction (e.g. a “like”) on acomment, replying to the comment, selecting an option to read replies tothe comment, or selecting the comment or user profile corresponding tothe comment. The media server may identify comments with a highestnumber of interactions as candidate comments. The media server may thensearch for text strings within the candidate comments, such as byidentifying nouns, proper nouns, or noun phrases in the comments and/orusing any of the aforementioned techniques, and select the identifiedtext strings as candidates for generating interactable elements.

Identifying text strings repeated in replies to comments may comprisecomparing text strings in the comment to text strings in replies to thecomment. The media server may be configured to identify text stringsthat are repeated a highest number of times in the replies to thecomments. The media server may additionally identify text strings basedon text string length or uniqueness across other text comments toidentify candidates for generating interactable elements, such as byusing any of the aforementioned methods.

Identifying previously searched terms may comprise storing dataidentifying search terms entered into a graphical user interface whensearching for other media assets. For example, each time a search isperformed for media assets, the media server may store data identifyingthe search terms and/or incrementing a value indicating a number oftimes a search term was used. In an embodiment, only a subset of thesearch terms entered into the interface are identified as previoussearch terms, such as search terms that have been repeated a thresholdnumber of times across searches and/or a top percentage, such as top tenpercent, of search terms based on a number of times the search termshave been repeated across searches. The media server may iterate throughwords or groupings of words and compare the identified words to theprevious search terms. Words or groupings of words that match the storedprevious search terms may be identified as candidates for generatinginteractable elements.

Identifying matching products may comprise searching for a product in astored data structure and/or third party search. For example, the mediaserver may store data identifying a plurality of products and comparethe text strings to the stored data to identify text strings that matcha name of a product. Additionally or alternatively, the media server mayenter the text strings into an external search, such as through an APIof a product listing application, and request search results for thetext strings. The media server may identify text strings that generatedsearch results generally, generated a threshold number of searchresults, and/or generated search results with a highest determinedrelevance.

Identifying locations may comprise searching for a location in a storeddata structure and/or third party search. For example, the media servermay store data identifying a plurality of locations and compare the textstrings to the stored data to identify text strings that match a name ofa location. Additionally or alternatively, the media server may searchenter the text strings into an external search, such as through an APIof a mapping or travel application, and request search results for thetext strings. The media server may identify text strings that generatedsearch results generally, generated a threshold number of searchresults, and/or generated search results with a highest determinedrelevance.

In some embodiments, a subset of the candidates generated using themethods described herein may be used to generate the interactableelements. For example, the candidates may be ranked based on candidatescore, length of text string, uniqueness of text string, prevalence oftext string in comments and/or replies, a number of interactions on thecomment, a number of search results for the text string, a number oftimes the text string was used in a search, or any combination thereof.Based on the ranks, a subset of the text strings may be selected. Forexample, the media server may select a predetermined number of textstrings, such as multiplier of the number of comments, thereby ensuringa similar distribution of interactable elements among different types ofvideos. Additionally or alternatively, the media server may select textstrings with a ranking above a threshold value and/or in a toppercentage of rankings.

In an embodiment, the text comments are pre-filtered to remove specificterms from analysis to be interactable elements. The specific terms mayinclude articles, common responses, or other ubiquitous data.Additionally or alternatively, the specific terms may include termsidentified in metadata of the media asset and/or terms used to searchfor recommended media assets. For example, the media server may comparetext strings to terms in the metadata of the media asset and onlygenerate interactable elements for terms that do not match the metadata.Thus, in FIG. 2 , the metadata of the media asset includes the term“Fortnite” in the description, but not “Minecraft.” Thus, in the firstcomment, “Fortnite” is not identified as a candidate for generatinginteractable elements, but “Minecraft” is. By only generatinginteractable elements from terms that are not listed in the metadata ofthe media asset and/or not used as part of the search for therecommended media assets, the media server removes duplicated resultsbetween the comment text and the media asset recommendations. Forinstance, while the term “Fortnite” may have been used to identify amedia asset recommendation based on the existence of the term in thedescriptor of the video, the term “Minecraft” may not have been used toidentify a media asset recommendation as the term was not in any of themetadata of the video.

In an embodiment, the media server identifies text strings in thecomments based on a determination that the comment contains additionalinformation, such as based on metadata corresponding to the device onwhich the comment was generated. For instance, the media server maydetermine that a comment contains additional information based on adetermination that a commentor has expertise in a topic of the commentor media asset, an identified location having been visited by acommentor, specific versions of general terms, a number of reactions tothe comment, previous comments by the commentor matching metadata of themedia asset, and/or data defining an amount of the media asset consumedby the commentor. If the media server determines a comment containsadditional information, the media server may analyze the comment fortext strings to generate interactable elements using methods describedherein.

In an embodiment, determining that the comment contains additionalinformation comprises determining that a user corresponding to a userprofile that posted the comment has expertise in a topic of the textcomment or the media asset. The topic of the media asset may bedetermined based on the title and/or other metadata of the video. Thetopic of the text comment may be determined based on text strings in thetext comment, such as by identifying unique terms in the text comment.The media server may determine that the user corresponding to the userprofile that posted the comment has expertise in the topic of thecomment or media asset based on a search history corresponding to a userprofile, a watch history corresponding to the user profile, applicationusage history of the user profile, and/or comment history of the userprofile. For example, if videos previously posted by the user profilecontain words pertaining to the topic, the media server may determinethat the user has expertise in the topic.

In an embodiment, determining that the comment contains additionalinformation comprises determining, from device metadata and/orapplication usage history, that a user corresponding to the user profilethat posted the comment has been to a location identified in thecomment. For example, metadata of a device may identify previouslocations visited by a user of the device or a map application mayprovide previous travel history of the device. The media server maycross-reference the text of the comment with visited locations todetermine whether any of the text matches a visited location. If any ofthe text matches a visited location, the media server may determine thatthe comment containing the text contains additional information and mayselect the text to generate an interactable element. For example, inFIG. 2 , the media server may identify the location of “Akihabara” as alocation where a device corresponding to the profile of “WorldTraveler”was identified.

In an embodiment, determining that the comment contains additionalinformation comprises determining a feature of a profile of thecommentor, such as whether the commentor profile is associated with apublic figure and/or is an uploader of popular videos. In someembodiments, the feature of the commentor is used to perform thesearches for identifiers of other media assets. For example, if afeature of the profile is that the profile has uploaded right-leaningpolitical videos, media assets identified for a text string thatidentifies a political figure may cause display of media assets with aright-leaning perspective of the political video.

In an embodiment, determining that the comment contains additionalinformation comprises identifying one or more specific terms that modifyone or more general terms. For example, the media server may identify ageneral term in a plurality of comments, such as the term “gardens.” Themedia server may determine if any of the comments contain specific termsthat modify the term “gardens,” such as the term “zen” or “tea” beforethe word “gardens.” If a comment contains the more specific terms, themedia server may determine that the comment contains additionalinformation.

In an embodiment, determining that the comment contains additionalinformation comprises determining that a number of reactions to thecomment exceeds a threshold or is within a top number or percentage ofnumber of reactions received by comments for the media asset. Thus, thecomments with the most reactions may be identified as containingadditional information.

In an embodiment, determining that the comment contains additionalinformation comprises comparing previous comments made by the commentoruser profile with metadata of the media asset, such as a title of themedia asset or tags of the media asset. If a threshold number ofprevious comments made by the commentor profile match the metadata ofthe media asset, the media server may determine that the commentcontains additional information.

In an embodiment, determining that the comment contains additionalinformation comprises determining that more than a threshold amount ofthe media asset was consumed by the device from which the comment wasreceived. For example, the media server may track an amount of a mediaasset played by the device prior to the device transmitting the commentto the media server. If the tracked amount of the media asset played bythe device does not exceed a threshold, such as half of the media asset,the media server may determine that the comment does not containadditional information.

In an embodiment, the media server stores identified text strings asmetadata for the media asset. The stored text strings may be the textstrings used to generate interactable elements, a subset of the textstrings used to generate interactable elements, such as the text stringswith the most selections, or other text strings in comments that wereidentified as containing additional information. In this manner, thecomments may be used to improve the metadata for a particular mediaasset.

In some embodiments, the media server additionally identifies timestampswithin the comments. For example, in FIG. 2 the second comment includesa timestamp of 7:25 at the end of the comment. The media server may beconfigured to search for text strings in a comment based on theidentification of the timestamp. Identification of the timestamp may beperformed by identifying numbers in the form of a timestamp (e.g.,##:##) and/or identifying a timestamp link entered into the comment. Inan embodiment, the media server is further configured to store dataassociating the identified text string with the timestamp. For instance,the media server may store data associating the text string of“Monokuma” with the timestamp of 7:25.

FIG. 3 depicts an example embodiment of a user interface where text incomments is replaced with an interactable element. In interface 300 ofFIG. 3 , the text comments 302 comprise interactable elements 304. Theinteractable elements 304 replace and/or are provided in conjunctionwith the text strings from which the interactable elements 304 weregenerated. For example, the interactable element may be designed to takethe appearance of the text string or to be overlayed over or near thetext string. In an embodiment, the display of the text string is alteredto indicate that the text string contains an interactable element, suchas by changing the color of the text string, highlighting the textstring, changing a font of the text string, or applying additionallyeffects, such as bold, underline, or shadow, to the text string.

The interactable element comprises an interface display which performsone or more functions when an interaction with the interactable elementis detected. The interaction may include any of a selection of theinteractable element, placement of a mouse cursors over the interactableelement, selection of the comment that includes the interactableelement, and/or any other method of detecting interaction with an optionin a graphical user interface.

In an embodiment, the function performed in response to the interactionwith the interactable element comprises navigating to a webpage orapplication where a search is performed for the text string. Forexample, the interactable element may comprise a hyperlink whichincludes a destination address and/or a term to be searched at thedestination address. When the interactable element is selected, the userdevice may navigate to a new webpage and/or open a new window, tabwithin a window, or an application designated by the hyperlink. The newwindow, tab, or application may include a search function. When the newwindow, tab, or application is opened, the search may be performed usingthe interactable element. For example, if the interface with the mediaassets are displayed through a browser, a new browser tab may be openedwith a search for other media assets provided by the media server.Additionally or alternatively, the new window, browser, or applicationmay be provided by an entity other than the media server, such as aninformation search interface or a product search interface.

In an embodiment, the function performed in response to the interactionwith the interactable element comprises displaying, on a same interfaceas the first media asset, identifiers of media assets corresponding tothe text string. For example, the media server may perform a search formedia assets provided by the media server using the text string. Thesearch may be performed when the interactable element is generated orwhen the interactable element is selected. The media server may displayone or more identifiers of media assets identified as a result of thesearch in a window displayed over the graphical user interface inresponse to the interaction with the interactable element. The windowmay be displayed on or near the text string, such that the window doesnot completely obscure the interface. Thus, the window which includesthe identifiers of the media assets may be displayed in conjunction withidentifiers recommended media assets and/or the viewed media asset.

In an embodiment, the searches described herein utilize additionalinformation in addition to using the text string as a search term. Theadditional information may include metadata relating to the displayedmedia asset, metadata relating to the text comment, and/or metadatarelating to the device on which the selection is made. The additionalinformation may be added to the search query, used to filter results ofthe search query, and/or used to sort results of the search query. Forexample, the media server may perform a search for the identified textstring, such as “Minecraft,” and then filter the results based on otherterms in the text comment, such as “Fortnite.”

Metadata relating to the displayed media asset which is used asadditional information may include terms in the title of the mediaasset, generated tags for the media asset, terms in a description of themedia asset, profile information of an account through which the mediaasset was posted, and/or data relating to other media assets which havehistorically been consumed by devices that consumed the displayed mediaasset. The profile information of the account may include a profilename, a profile description, demographic data of the account, othermedia assets posted by the account, media assets identified to besimilar to other media assets posted by the account, accounts identifiedas being similar to the account, and/or media assets consumed by theaccount.

Metadata relating to the text comment may include other text in the textcomment, profile information of the commentor account, or text ofreplies to the text comment. The profile information of the commentoraccount may include a profile name, a profile description, demographicdata of the account, other media assets consumed by the account, othermedia assets on which the account provided comments, accountscorresponding to other media assets consumed by the account, accounts towhich the commentor account is subscribed, or any other additionaltracked data relating to a device of the commentor account, such aspurchase history, travel history, or search history.

Metadata relating to the device on which the selection is made mayinclude profile information of the viewing account. The profileinformation of the viewing account may include a profile name, a profiledescription, demographic data of the account, other media assetsconsumed by the account, other media assets on which the accountprovided comments, accounts corresponding to other media assets consumedby the account, accounts to which the viewing account is subscribed, orany other additional tracked data relating to a device of the viewingaccount, such as purchase history, travel history, or search history.

FIG. 4 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes simultaneous display of amedia asset, identifiers of recommended media assets, and identifiers ofmedia assets identified based on the text of the multimedia link. Ininterface 400 of FIG. 4 , the text comments 402 comprise interactableelements 404. A cursor 406 is depicted interacting with one of theinteractable elements 404. The interaction may include hovering over theinteractable element or input selecting the interactable element, suchas a click of the interactable element.

In response to the interaction with the interactable element, interface400 displays window 408. Window 408 is displayed as an overlay overinterface 400. The window includes identifiers 410 of other media assetsidentified through the search. For example, the search for videos withthe search term “Minecraft” produced the two results depicted in window408. Additional results may be displayed in window 408 in response to aselection of the down-arrow. By depicting identifiers of media assetsrelating to terms in the comments in a window overlay, the systemprovides an uncluttered user interface which provides options fordisplaying relevant videos, as different selections may cause display ofdifferent media asset identifiers based on the selected text string. Inresponse to selection of one of the identifiers 410, the media servermay cause the viewer device to navigate to a new page where the mediaasset corresponding to the identifier is displayed. Additionally oralternatively, the media asset may be displayed in a window, such aswindow 408, displayed on interface 400.

In an embodiment, displaying the identifiers of the media assetscorresponding to the text string comprise displaying the media assetsalong with the recommended media assets. For example, the media servermay perform a search for media assets based on the text string anddisplay identified media assets along with the identifiers ofrecommended media assets. Additionally or alternatively, the mediaserver may use the selection to filter or reorder the identifiers of therecommended media assets. For example, in response to a selection of“Minecraft,” the media server may identify recommended videos thatinclude “Minecraft” in the title or metadata and move the identifiedvideos to the top of the list of identifiers of recommended videos.

FIG. 5 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes display of identifiers ofmedia assets based on the text of the multimedia link to be displayed ina recommendations section of the user interface. In interface 500 ofFIG. 5 , the text comments 502 comprise interactable elements 504. Acursor 506 is depicted interacting with one of the interactable elements504. The interaction may include hovering over the interactable elementor input selecting the interactable element, such as a click of theinteractable element.

In response to the interaction with the interactable element 504,interface 500 depicts identifiers 510 of media assets identified througha search based on the text string along with the identifiers 508 ofrecommended media assets. Identifiers 510 may comprise identifiers thatwere initially a part of recommended media assets. For example, themedia server may search through the recommended media assets for anymedia assets which include the term “Minecraft” in response to theselection of the interactable element for “Minecraft” and cause theresulting media assets to be displayed prior to other media assets, suchas by moving any identified media assets to the top of a list,displaying the identified media assets in a first page of media assets,or otherwise prioritizing display of the identified media assets.Additionally or alternatively, the media server may perform a search formedia assets based on the selection of “Minecraft” and populate at leasta portion of the recommended media assets with the search results.

While FIG. 5 depicts the searching, filtering, or sorting beingperformed in response to a selection of the interactable element in thetext string in the text comment, in other embodiments the identifiedtext string is used to generate a separate filter or sort option. Forexample, the media server may display a plurality of sort or filteroptions above the identifiers of the recommended media assets. One ormore of the options may be generated in response to identifying the textstring in the text comment. Additionally or alternatively, in responseto a selection of the text string in the text comment, the interface 500may update the plurality of sort or filter options to include theidentified text string.

In an embodiment, the function performed in response to the interactionwith the interactable element comprises causing display of comments thatinclude the text string. For example, the interface 500 may sort orfilter the comments based on the selected text string such that commentsthat include the text string or have replies that include the textstring are displayed first. As with the filter of the media assets, theoption to sort or filter the comments based on the text string may beincluded in the interactable element in the text string in the commentand/or as a separate filter or sort option generated based onidentifying the text string in the comment.

In an embodiment, the interactable text strings are used to determine anordering of the comments. For example, comments that containinteractable text strings may be prioritized over comments that do notcontain interactable text strings. As another example, comments maybesorted based on a number of interactions with interactable text stringsin the comments. For instance, comments with text strings that have ahighest number of interactions may be sorted with comments that have ahighest number of interactions with the interactable elements beingdisplayed first. Additionally or alternatively, the sorting may beweighted based on a type of interaction. For example, moving a cursorover the interactable element may be weighted lower than selecting anidentifier of another media asset based on the interactable element.

FIGS. 4 and 5 depict embodiments where the identifiers of media assetscomprise identifiers of media assets provided by the media server. Thus,the searches performed for FIGS. 4 and 5 comprise searches of a datarepository identifying media assets provided by the media server. Inother embodiments, the media server may be configured to send a searchquery to one or more external servers, such as through an API of anapplication provided by the one or more external servers or through asearch interface provided by the one or more external servers, where thesearch query comprises the text string as a search term. Additionally oralternatively, the media server may store data identifying media assetsof the external server and may perform a search through the stored data.

The search results for media assets provided by external servers, suchas through different platforms, may be provided in a similar interfaceas depicted in FIG. 4 or 5 . For instance, the media server may displayidentifiers of media assets identified from searches through externalplatforms. The identifiers may be displayed along with identifiers ofmedia assets identified through a search of media assets provided by themedia server.

FIG. 6 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes display of identifiers ofmedia assets hosted by different platforms. In interface 600 of FIG. 6 ,the text comments 602 comprise interactable elements 604. A cursor 606is depicted interacting with one of the interactable elements 604. Theinteraction may include hovering over the interactable element or inputselecting the interactable element, such as a click of the interactableelement.

In response to the interaction with the interactable element, interface600 displays window 608. Window 608 is displayed as an overlay overinterface 600. The window includes an identifier 610 of a media assetidentified through a search through media assets provided by the mediaserver. The window additionally includes an identifier 612 of a mediaasset provided by a first external platform and an identifier 614 of amedia asset provided by a second external platform. Identifier 612comprises an identifier of a product page where a product correspondingto the text string can be purchased. Identifier 614 comprises anidentifier of an information page corresponding to the text string.Identifier 612 may be identified through a search of products providedby the RetailShopper platform. Identifier 614 may be identified througha search of information pages provided by the Learning Nexus platform.Additionally or alternatively, both identifiers may be identifiedthrough a search of a general repository that identifies pages providedby both the RetailShopper platform and the Learning Nexus platform.

In an embodiment, the identifiers of media assets provided by externalplatforms may comprise hyperlinks to websites corresponding to thesearch results. Thus, in response to a selection of an option from anexternal platform, the viewing device may navigate to a pagecorresponding to the external platform. For example, in response to aselection of identifier 614, the viewing device may navigate to a webpage hosted by Learning Nexus which corresponds to “Monokuma.”Additionally or alternatively, the media server may use an API providedby an external platform to provide access to the platform functionalityfrom the interface. The navigation to the web page may be performed inwindow 608, in a separate window or tab, or as a navigation from a webpage that include interface 600.

The media server may request an interface from the external platformthrough the API and cause display of the interface from the externalplatform in the same window or in a separate window. For example, inresponse to a selection of identifier 612, the media server replaceidentifier 612 with an image of the product provided by the externalserver with additional information, an option to complete a purchase,and an option to view a full webpage. If the option to complete thepurchase is selected, the media server may facilitate authentication ofa user account through the API, such as providing known credentials, orthe external server may authenticate the user directly and complete thetransaction. As the transaction is completed through the displayedwindow, the transaction does not interrupt a media asset viewingsession.

In an embodiment, identifiers of media assets provide from externalsources have different visual indicators than identifiers of mediaassets provided by the media server. For instance, different icons aredepicted in FIG. 6 for video assets, the product platform, and theinformation platform. Additionally or alternatively, other visualindications may be provided to distinguish between the differentplatforms. For example, the identifiers of media assets from the mediaserver may have a border of a first color, identifiers of media assetsfrom an external server through which information is provided through anAPI may have a border of a second color, and identifiers of media assetsfrom an external server where the identifier comprises a hyperlink to adifferent website may have a border of a third color. In this manner,the interface may distinguish between options which will cause thedevice to navigate away from interface 600, such as through a hyperlinkto a different website, and options that will cause the device todisplay additional information or windows through interface 600.

FIG. 7 depicts an example embodiment of a user interface where inputinteraction with a displayed identifier of a media asset hosted by adifferent platform causes display of an additional window correspondingto the different platform. In interface 700 of FIG. 7 , the textcomments 702 comprise interactable elements 704. A cursor 706 previouslyinteracted with one of the interactable elements 704. The interactionmay include hovering over the interactable element or input selectingthe interactable element, such as a click of the interactable element.

In response to the interaction with the interactable element, interface700 displayed window 708. Window 708 is displayed as an overlay overinterface 700. The window includes an identifier 710 of a media assetidentified through a search through media assets provided by an externalplatform. The cursor 706 is depicted as interacting with the identifier710 of the media asset identified through the search through mediaassets provided by the external platform. In response to the selection,interface 700 displays external platform window 712. External platformwindow 712 comprises a window where contents of the window are providedby an external platform, such as through a website or through an API ofthe external platform. In the example, the external platform comprises aproduct platform with options for reviewing a product and for purchasingthe product.

In some embodiments, the media server is configured to search throughspecific external platforms for each identified text string. In otherembodiments, the media server performs a general search through a searchinterface which searches a plurality of different platforms.Additionally or alternatively, the media server may be configured tosearch platforms based on a type of text string identified. For example,in response to detecting a text string that corresponds to a location,the media server may be configured to search a travel listing platformand/or a maps platform.

FIG. 8 depicts an example embodiment of a user interface where inputinteracting with a multimedia link causes display of identifiers ofmedia assets based on an identified location. In interface 800 of FIG. 8, the text comments 802 comprise interactable elements 804. A cursor 806is depicted interacting with one of the interactable elements 804. Theinteraction may include hovering over the interactable element or inputselecting the interactable element, such as a click of the interactableelement.

In response to the interaction with the interactable element, interface800 displays window 808. Window 808 is displayed as an overlay overinterface 800. In an embodiment, in response to the text stringcontaining an identifier of a location, window 808 includes identifiers810, 812, and 814 which correspond to platforms which utilize locations.Selection of the identifiers may cause a user device to executeapplications corresponding to the identifiers, the user device tonavigate to a webpage corresponding to the identifiers, or cause theinterface to display, through window 808 or a separate window, dataretrieved from an external server. Identifier 810 comprises anidentifier of the location as identified through a map platform. Thus, aselection of identifier 810 may cause display of the identified locationin the map, such as through window 808 or a separate window provided bythe media server. Identifier 812 comprises an identifier of a travellisting page for booking a trip to the identified location. Thus, aselection of identifier 812 may cause the user device to display optionsto book a trip to the identified location through a travel listingapplication or website, such as through window 808 or a separate windowprovided by the media server.

While FIG. 2-8 generally describe display of identifiers of mediaassets, other application integrations may be utilized based onidentified text string. For example, integration with a dictionaryapplication may be utilized to provide a definition for the text string,integration with a translation application may be utilized to provide atranslation for the text string, and integration with a planningapplication may be utilized to store data in association with a newplan. As an example of a planning application, a selection of aningredient name may cause the media server to identify an applicationfor a shopping list and provide an option to add the ingredient to theshopping list. As another example, identifier 814 in FIG. 8 comprises anidentifier of an option to add a destination to an itinerary. In someembodiments, the media server may determine, based on itineraryinformation provided by a calendar application or travel informationprovided by a travel listing application, that a trip has been booked toa region that includes the location and, in response, may provide anoption to add a stop at the location. For example, if the media serverdetermines that a trip has been planned to Japan, window 808 ininterface 800 may include the option to add a stop at Akihabara to anitinerary in response to a selection of the text string “Akihabara” inthe text comment.

Embodiments described herein with respect to FIG. 2-8 depict the use ofidentified text strings in text comments to generate interactableelements in the text comments. These embodiments provide substantialimprovements to the graphical user interface by generating new ways ofdisplaying other media asset viewing options, integrating the displaywith external platforms, and leveraging additional information in asearch or filtering of videos, such as commentor profile information orviewer profile information. The methods described herein mayadditionally be used to generate recommendations. For example, the mediaserver may identify text strings using the methods described herein anduse the identified text strings to generate recommendations for mediaassets. The recommendations may be based on a prevalence of a textstring, a number of interactions with the text string, and/or viewerinteraction with the text string. For example, text comments with themost interactions may be analyzed to identify text strings which areused to search for recommended content. As another example, the mediaserver may augment recommendations for a user or user profile preferenceinformation based on detecting an interaction with a text comment andidentifying a particular text string within the text comment.

In some embodiments, text stings are identified for generatingrecommendations for media assets based, at least in part, oninteractions with text comments that include the text strings. Forexample, the media server may identify text comments that received athreshold number or highest number of replies, text comments thatreceive a threshold number or highest number of interface reactions,text comments that are displayed on one or more devices for greater thana threshold amount of time or for a highest amount of time, textcomments over which a cursor has hovered for greater than a thresholdamount of time or for a highest amount of time, and/or text commentsthat have a portion that has been copied more than a threshold number oftimes or a highest amount of times. “Highest,” as used in this section,refers to one of a threshold number of top values. For example, themedia server may identify comments that received the top five mostinterface interactions.

In some embodiments, the text strings in the text comments are used toaugment display of the media asset corresponding to the text comments.For example, the media server may analyze the text strings in the textcomments to identify a particular text string, such as a text stringthat maximizes one of the values described previously described herein.In an embodiment, the selected text string is one that is not identifiedin the metadata of the media asset. For example, the text string of“Minecraft” may be identified due to the term not being included inmetadata of the media asset and due to a high volume of comments thatinclude the term “Minecraft,” a high number of interactions on thecomment that includes the term “Minecraft,” or an interaction by theviewing device with the comment including the term or the text string ofthe term itself. The identified particular text string may be used toidentify supplemental content such as through a search of supplementalcontent stored by the media server and/or provided by an external servercomputer. The media server may augment display of the media asset withthe supplemental content.

In an embodiment, augmenting the display of the media asset with thesupplemental content comprises displaying the supplemental contentbefore the media asset, after the media asset, or after pausing playbackof the media asset. As a practical example, in response to identifyingthe text string “Minecraft” in the comments, the media server mayrequest an advertisement for Minecraft from an advertisement server andcause the advertisement to play prior to the video.

In an embodiment, augmenting the display of the media asset with thesupplemental content comprises displaying an overlay over the mediaasset. For example, the media server may identify and/or generate abanner for display at the top of a video. As a practical example, inresponse to identifying the “Monokuma” text string, the media server maygenerate a banner advertisement for a Monokuma doll. The media servermay cause the banner advertisement to display over the media asset.

In an embodiment, the media server identifies a time within the mediaasset to augment the display of the media asset based on the text stringor other information in the comment. In some embodiments, the mediaserver identifies timestamps in the text comment and uses the timestampsto determine a time within playback of the media asset to augment themedia asset. For example, the comment that includes the term “Monokuma”additionally includes a timestamp of 7:25. Based on the identifiedtimestamp, the media server may cause display of the supplementalcontent at or near a point in time corresponding to the timestamp. Forinstance, if the video includes a first advertisement break at threeminutes and a second advertisement break into the video, the mediaserver may identify the second advertisement break at eight minutes asbeing closest to the timestamp of 7:25 and cause display of thesupplemental content at a second advertisement break.

In some embodiments, the media server identifies other terms in the textcomment and uses the other terms to determine a time within playback ofthe media asset augment the media asset. For example, in the firstcomment, the media server may identify the term “Fortnite” as anotherterm in the comment including the text string “Minecraft.” Based on theterm “Fortnite,” the media server may identify a portion of the videocorresponding to the term “Fortnite,” such as based on metadata of thevideo, analysis of the description of the video, analysis of the audiocorresponding to the video, or analysis of subtitles of the video toidentify times in the media asset when the term “Fortnite” is mentioned.For instance, the video description identifies a section relating toFortnite starting at a timestamp of 3:42. Thus, the media server mayidentify the timestamp of 3:42 or a timestamp close to the timestamp of3:42 for placing supplemental content relating to Minecraft.

In some embodiments, the media server analyzes audio of the media asset,subtitles for the media asset, and/or images of the media asset toidentify a timestamp for the supplemental content. For example, themedia server may identify subtitles that match the term “Akihabara” anddetermine a timestamp corresponding to the matching subtitles. Asanother example, the media server may identify an image corresponding tothe term “Monokuma,” such as through a search of an image repositoryand/or through an image of the supplemental content corresponding to theterm “Monokuma.” The media server may employ known image analysistechniques to identify an image in the video that matches the identifiedimages corresponding to the term “Monokuma.” The media server may usethe timestamp of the identified image in the video to identify thetimestamp for the supplemental content.

In some embodiments, the media server additionally analyzes the mediaasset to identify locations in the media asset for the supplementalcontent. For example, based on the identification of the image of“Monokuma” in the video, the media server may identify the locationcorresponding to the image as being a location for displayingsupplemental content, such as a banner advertisement for a Monokumadoll. The media server may be configured to use timestamps or additionalinformation in the text comments to determine which frames of the videoto analyze. For example, for the “Minecraft” supplemental content, themedia server may analyze frames of the video starting at 3:42 due to theinclusion of the term “Fortnite” in the comment and the indication of atimestamp corresponding to “Fortnite” in the video and/or due to audioor subtitles that include the term “Fortnite.” Similarly, for the“Monokuma” supplemental content, the media server may analyze frames ofthe video at or near the timestamp of 7:25 based on the timestampidentified in the comment.

In some embodiments, augmenting the display of the media asset comprisesmaking one or more portions of the media asset interactable. Forexample, the media server may analyze frames of a video to identify alocation of a particular item corresponding to the supplemental content,such as a Monokuma doll. The media server may augment display of themedia asset such that the location of the particular item at thetimestamps that include the particular item contains an interactableelement. In response to an interaction with the interactable element,the media server may display the supplemental content.

FIG. 9 depicts an example embodiment of modifying display of a mediaasset with supplemental content based on data from a text comment. Ininterface 900 of FIG. 9 , the text comments 902 comprise identified textstrings 904. The second of the identified text strings 904 is used toidentify an image 906 in the video. An interactable element is overlaidover image 906. In response to a selection of the interactable elementoverlaid over image 906, the interface 900 displays supplemental content908. Supplemental content 908, in the example of FIG. 9 comprises anoption to purchase a product through an external server. Thesupplemental content 908 may comprise a static element, such as adescription of the item, or a dynamic element which reacts similarly toinput as FIG. 7 's identifier 710.

FIG. 10 depicts an example method for generating interactable textstrings in comments based on text analysis.

At step 1002, input/output circuitry of a server computer transmits amedia asset to one or more devices. For example, the input/outputcircuitry of the server computer may provide a graphical user interfaceto one or more devices through which a media asset, such as a video,image, audio recording, or text, can be displayed.

At step 1004, input/output circuitry of the server computer receivestext comments from the one or more devices. For example, theinput/output circuitry may provide, within the graphical user interface,an option to generate one or more text comments that are related to themedia asset. The input/output circuitry of the server computer mayreceive selections of the option to generate the one or more textcomments from a plurality of user devices.

At step 1006, control circuitry of the server computer selects a textstring in the text comments. For example, the control circuitry of theserver computer may identify a plurality of text strings in the textcomments. The text strings may be identified as full words within thecomment and/or combinations of words in the comment. For instance, themedia server may be configured to analyze text comments based on n-gramsof varying size, thereby encompassing single words and a plurality ofwords. The media server may additionally employ other text analysistechniques to ensure that only particular groupings of terms areanalyzed. For example, the media server may be configured to separateverbs from noun phrases or to ignore specific terms, such as articles.

At step 1008, control circuitry of the server computer determineswhether the text string should be interactive. For example, the controlcircuitry of the server computer may determine whether the text stringshould be interactive using any of the methods described herein. Thedetermination may be based on a comparison of scores between other textstrings. Thus, step 1008 may be performed after a plurality of textstrings in the comments have been identified and pre-analyzed togenerate the score. The determination may additionally include adetermination that the text string is different from metadata of themedia asset, such as based on a comparison between the text string andmetadata elements of the media asset. The text string may be determinedto be different from the metadata of the media asset if a search throughthe metadata of the media asset does result in identification of thetext string in the metadata of the media asset. If control circuitry ofthe server computer determines that the text string should not beinteractive, the process proceeds to step 1012, which is describedfurther herein.

If at step 1008, control circuitry of the server computer determinesthat the text string should be interactive, at step 1010, controlcircuitry of the server computer modifies the text string to contain aninteractive element. For example, control circuitry of the servercomputer may add a hyperlink into the text comment and/or otherinteractable elements such that the media server receives an indicationwhen an interaction is received with the portion of the text commentcontaining the text string and can perform one or more actions describedpreviously herein in response.

At step 1012, control circuitry of the server computer determines ifthere are more text strings to analyze. If control circuitry of theserver computer determines there are more text strings to analyze, theprocess returns to step 1006 and a new text string is selected. Ifcontrol circuitry of the server computer determines there are notanymore text strings to analyze, the process ends.

FIG. 11 depicts an example method for generating interactable textstrings in comments based on a determination that a comment containsadditional information.

At step 1102, input/output circuitry of a server computer transmits amedia asset to one or more devices. For example, the input/outputcircuitry of the server computer may provide a graphical user interfaceto one or more devices through which a media asset, such as a video,image, audio recording, or text, can be displayed.

At step 1104, input/output circuitry of the server computer receivestext comments from the one or more devices. For example, theinput/output circuitry may provide, within the graphical user interface,an option to generate one or more text comments that are related to themedia asset. The input/output circuitry of the server computer mayreceive selections of the option to generate the one or more textcomments from a plurality of user devices.

At step 1106, control circuitry of the server computer identifies atopic of the media asset. For example, control circuitry may determine atopic from a media asset title, media asset tags, or other metadata ofthe media asset.

At step 1108, control circuitry of the server computer selects a textcomment. For example, the control circuitry may select one of theplurality of text comments generated for the media asset to analyze.

At step 1110, control circuitry of the server computer determineswhether the comment contains additional information relating to thetopic of the media asset. For example, the control circuitry of theserver computer may analyze metadata of a device from which the textcomment was received to determine whether the comment containsadditional information relating to the topic of the media asset, such asbased on previous comments left by a profile corresponding to thedevice, locations visited by the device, or other media assets viewed bythe device. In other embodiments, determining that the comment containsadditional information comprises determining that the text commentreceived more than a threshold number of reactions or determining thatthe text comment contains specific terms modifying general terms.

If at step 1110, control circuitry of the server computer determinesthat the text comment does not contain additional information, theprocess proceeds to step 1114, which is described further herein. If, atstep 1110, control circuitry of the server computer determines that thetext comment contains additional information, at step 1116, controlcircuitry of the server computer modifies a text string of the textcomment to contain an interactive element. For example, controlcircuitry of the server computer may identify a text string in the textcomment using the methods described herein and add a hyperlink into thetext comment and/or other interactable elements such that the mediaserver receives an indication when an interaction is received with theportion of the text comment containing the text string and can performone or more actions described previously herein in response.

At step 1114, control circuitry of the server computer determines ifthere are more text comments to analyze. If control circuitry of theserver computer determines there are more text comments to analyze, theprocess returns to step 1108 and a new text comment is selected. Ifcontrol circuitry of the server computer determines there are notanymore text comments to analyze, the process ends.

FIG. 12 depicts an example method for modifying media assets based ontext strings in text comments.

At step 1202, input/output circuitry of a server computer transmits amedia asset to one or more devices. For example, the input/outputcircuitry of the server computer may provide a graphical user interfaceto one or more devices through which a media asset, such as a video,image, audio recording, or text, can be displayed.

At step 1204, input/output circuitry of the server computer receivestext comments from the one or more devices. For example, theinput/output circuitry may provide, within the graphical user interface,an option to generate one or more text comments that are related to themedia asset. The input/output circuitry of the server computer mayreceive selections of the option to generate the one or more textcomments from a plurality of user devices.

At step 1206, control circuitry of the server computer selects a textstring in the text comments. For example, the control circuitry of theserver computer may identify a plurality of text strings in the textcomments. The text strings may be identified as full words within thecomment and/or combinations of words in the comment. For instance, themedia server may be configured to analyze text comments based on n-gramsof varying size, thereby encompassing single words and a plurality ofwords. The media server may additionally employ other text analysistechniques to ensure that only particular groupings of terms areanalyzed. For example, the media server may be configured to separateverbs from noun phrases or to ignore specific terms, such as articles.

At step 1208, control circuitry of the server computer determineswhether the text string should be used to modify the media asset. Forexample, the control circuitry of the server computer may determinewhether the text string should be used to modify the media asset usingany of the methods described herein. The determination may be based on acomparison of scores between other text strings. Thus, step 1208 may beperformed after a plurality of text strings in the comments have beenidentified and pre-analyzed to generate the score. The determination mayadditionally include a determination that the text string is differentfrom metadata of the media asset, such as based on a comparison betweenthe text string and metadata elements of the media asset. If controlcircuitry of the server computer determines that the text string shouldnot be used to modify the media asset, the process proceeds to step1214, which is described further herein.

If at step 1208, control circuitry of the server computer determinesthat the text string should be used to modify the media asset, at step1210, control circuitry of the server computer identifies supplementalcontent based on the text string. For example, control circuitry of theserver computer may identify an overlay for a video that matches thetext string and/or identify a video asset, such as an advertisement,that matches the text string.

At step 1212, control circuitry of the server computer modifies themedia asset to display the supplemental content. For example, controlcircuitry of the server computer may insert a video advertisement thatmatches the text string into the display of the media asset and/or causedisplay of a media overlay during display of the media asset, such as ata point in time where an image matching the text string is beingdisplayed.

At step 1214, control circuitry of the server computer determines ifthere are more text strings to analyze. If control circuitry of theserver computer determines there are more text strings to analyze, theprocess returns to step 1206 and a new text string is selected. Ifcontrol circuitry of the server computer determines there are notanymore text strings to analyze, the process ends.

Example Implementation: Hardware and Software Overview

FIG. 13 depicts a generalized embodiment of an illustrative device(e.g., media server 120, user devices 130, viewer device 140). Theillustrative device may include a media server that transmits mediaassets to viewing devices, receives text comments, analyzes the textcomments, modifies the text comments to contain interactable elements,and causes display of the text comments with the interactable elementson a client device. The illustrative device may additionally include auser such as a smartphone, a tablet, personal computer, set-top box,etc. (discussed further below with respect to FIG. 14 ) which displaysmedia assets and text comments. Device 1300 may transmit media and/ortext comments through input/output (hereinafter “I/O”) path 1302. I/Opath 1302 may provide the media assets, text comments, or other data tocontrol circuitry 1304, which includes processing circuitry 1306 andstorage 1308. Control circuitry 1304 may be used to send and receivecommands, requests, and other suitable data using I/O path 1302. I/Opath 1302 may comprise circuitry that connects control circuitry 1304(and specifically processing circuitry 1006) to one or morecommunications paths (described below in relation to FIG. 14 ). I/Ofunctions may be provided by one or more of these communications pathsbut are shown as a single path in FIG. 13 to avoid overcomplicating thedrawing. I/O path 1302 may comprise circuitry for transmitting messagesover network 1414 for example, when directed by control circuitry 1304to transmit said messages. In some embodiments, I/O path 1302 maycomprise circuitry that includes wires and/or buses connected to aphysical network port, e.g. an ethernet port, a wireless WiFi port,cellular communication port, or any other type of suitable physicalport.

Control circuitry 1304 may be based on any suitable processing circuitrysuch as processing circuitry 1306. Processing circuitry should beunderstood to mean circuitry based on one or more microprocessors,microcontrollers, digital signal processors, programmable logic devices,field-programmable gate arrays (FPGAs), application-specific integratedcircuits (ASICs), etc., and may include a multi-core processor (e.g.,quad-core). In some embodiments, processing circuitry may be distributedacross multiple separate processor or processing units, for example,multiple of the same type of processing units (e.g., two Intel Core i7processors) or multiple different processors (e.g., Ryzen processor withintegrated CPU and GPU processing cores) or may include a multi-coreprocessor (e.g., dual-core, quad-core, hexa-core, or any suitable numberof cores) or supercomputer. In some embodiments, control circuitry 1004executes instructions for an application stored in memory (e.g., memory1308). In some implementations, any action performed by controlcircuitry 1304 may be based on instructions received from an applicationexecuting on device 1300.

The circuitry described herein, including, for example, tuning, audiogenerating, encoding, decoding, encrypting, decrypting, scaler, andanalog/digital circuitry, may be implemented using software running onone or more general purpose or specialized processors. If storage 1308is provided as a separate device from device 1300, the tuning andencoding circuitry may be associated with storage 1308.

Storage 1308 may be any device for storing electronic data, such asrandom-access memory, solid state devices, quantum storage devices, harddisk drives, non-volatile memory or any other suitable fixed orremovable storage devices, and/or any combination of the same. Controlcircuitry 1304 may allocate portions of storage 1308 for variouspurposes such as caching application instructions, recording mediaassets, storing portions of a media asset, buffering segments of media,etc. As described herein, storage 1308 may be used to store one or moreLUTs storing a number of MAC addresses associated with a plurality ofuser equipment devices and their corresponding profile information.

A user may send instructions to control circuitry 1304 using user inputinterface 1310. User input interface 1310 may be any suitable user inputinterface, such as a remote control, mouse, trackball, keypad, keyboard,touchscreen, touchpad, stylus input, joystick, voice recognitioninterface, or other user input interfaces. Instructions to controlcircuitry 1304 may be transmitted through I/O path 1302, which couldconsist of a video tracking and detection mechanism, Internet of Things(IoT) and home automation triggers, emergency alert systems, andsoftware or hardware communication pipelines and/or notificationcenters.

Display 1312 may be provided as a stand-alone device or integrated withother elements of each one of user equipment device 1300. For example,display 1312 may be a touchscreen or touch-sensitive display, aprojector, or a casting device. In such circumstances, user inputinterface 1310 may be integrated with or combined with display 1312.Display 1312 may be one or more of a monitor, a television, aliquid-crystal display (LCD) for a mobile device, silicon display, e-inkdisplay, light-emitting diode (LED) display, or any other suitableequipment for displaying visual images. Graphics processing circuitrymay generate the output to the display 1312. In some embodiments, thegraphics processing circuitry may be external to processing circuitry1306 (e.g., as a graphics processing card that communicates withprocessing circuitry 1306 via I/O path 1302) or may be internal toprocessing circuitry 1306 or control circuitry 1304 (e.g., on a samesilicone die as control circuitry 1304 or processing circuitry 1306). Insome embodiments, the graphics processing circuitry may be used toreceive, display, and play content.

Speakers 1314 may be provided as integrated with other elements of userequipment device 1300 or may be stand-alone units. The audio componentof videos and other content displayed on display 1312 may be playedthrough speakers 1314. In some embodiments, the audio may be distributedto a receiver (not shown), which processes and outputs the audio viaspeakers 1314. The speakers 1314 may be part of, but not limited to, ahome automation system.

Streaming applications may be, for example, stand-alone applicationsimplemented on user devices. For example, the streaming application maybe implemented as software or a set of executable instructions, whichmay be stored in non-transitory storage 1308 and executed by controlcircuitry 1304 of a user device 1402, 1404, 1406. In such an approach,instructions of the streaming application are stored locally (e.g., instorage 1308), and data for use by the application is downloaded on aperiodic basis (e.g., from an out-of-band feed, from an Internetresource, or using another suitable approach). Control circuitry 1304may retrieve instructions of the streaming application from storage 1308and process the instructions to generate any of the displays discussedherein. Based on the processed instructions, control circuitry 1304 maydetermine what action to perform when user input is received from inputinterface 1310.

FIG. 14 depicts an example media system in accordance with someembodiments of the disclosure, in which device 1300 can be implementedin the media system 1400 of FIG. 14 as user television equipment 1402,user computer equipment 1404, wireless user communications device 1406,or any other type of user equipment suitable for accessing media. Forsimplicity, these devices may be referred to herein collectively as userequipment or a user device. User equipment, on which the mediaapplication or the streaming application is implemented, may function asa stand-alone device or may be part of a network of devices. Variousnetwork configurations of devices may be implemented and are discussedin more detail below.

User television equipment 1402 may include a set-top box, an integratedreceiver decoder (IRD) for handling satellite television, a televisionset, a digital storage device, or other user television equipment. Oneor more of these devices may be integrated to be a single device, ifdesired. User computer equipment 1404 may include a PC, a laptop, astreaming content aggregator, a PC media center, or other user computerequipment. It may include devices like digital assistance, smartspeakers, and/or home automation. Wireless user communications device1406 may include a smartphone, a portable video player, a portable musicplayer, a portable gaming machine, a tablet, a wireless streaming deviceor other wireless device. It should be noted that the lines are blurredwhen trying to classify a device as one of the above devices and onedevice may be categorized into one or more of the categories listedabove.

In system 1400, there are typically more than one of each type of userequipment, but only one of each is shown in FIG. 14 to avoidovercomplicating the drawing. In addition, each user may utilize morethan one type of user equipment (e.g., a user may have a computer and atablet) and also more than one of each type of user equipment device(e.g., a user may have multiple television sets).

The user equipment may be coupled to communications network 1414.Namely, user television equipment 1402, user computer equipment 1404,and wireless user communications device 1406 are coupled tocommunications network 1414 via communications paths 1408, 1410, and1412, respectively. Communications network 1414 is used by the userequipment to obtain the video stream. Communications network 1414 may beone or more networks including the Internet, a mobile phone network,ad-hoc network, or other types of communications network or combinationof communications networks. Paths 1408, 1410, and 1412 may separately ortogether include one or more communications paths, including anysuitable wireless communications path. Path 1412 is drawn as a dottedline to indicate it is a wireless path. Communications with the userequipment may be provided by one or more of these communications pathsbut are shown as a single path in FIG. 14 to avoid overcomplicating thedrawing. The user equipment devices may communicate with each otherdirectly through an indirect path via communications network 1414.

System 1400 includes content item source 1416 and content item datasource 1418 coupled to communications network 1414 via communicationspaths 1420 and 1422, respectively. Paths 1420 and 1422 may include anyof the communications paths described above in connection with paths1408, 1410, and 1412. Communications with the content item source 1416and content item data source 1418 may be exchanged over one or morecommunications paths but are shown as a single path in FIG. 14 to avoidovercomplicating the drawing. In addition, there may be more than one ofeach of content item source 1416 and content item data source 1418, butonly one of each is shown in FIG. 14 to avoid overcomplicating thedrawing. In some embodiments, content item source 1416 and content itemdata source 1418 are integrated as one source device. Althoughcommunications between sources 1416 and 1418 with user equipment areshown as through communications network 1414, in some embodiments,sources 1416 and 1418 may communicate directly with user equipmentdevices 1402, 1404, and 1406 via communications paths (not shown) suchas those described above in connection with paths 1408, 1410, and 1412.

Content item source 1416 or content item data source 1418 may includeone or more types of media distribution equipment such as a mediaserver, cable system headend, satellite distribution facility,intermediate distribution facilities and/or servers, Internet providers,on-demand media servers, and other media providers. Content item source1416 or content item data source 1418 may be the originator of mediacontent or may not be the originator of media content. Content itemsource 1416 or content item data source 1418 may also include a remotemedia server used to store different types of media content (including amedia asset selected by a user), in a location remote from any of theuser equipment. Systems and methods for providing remotely stored mediato user equipment are discussed in greater detail in connection withEllis et al., U.S. Pat. No. 7,761,892, issued Jul. 20, 2010, which ishereby incorporated by reference herein in its entirety.

System 1400 is intended to illustrate a number of approaches, or networkconfigurations, by which user equipment devices and sources of mediacontent and guidance data may communicate with each other for thepurpose of accessing media and data related to the media. Theconfiguration of the devices and paths in system 1400 may change withoutdeparting from the scope of the present disclosure.

The processes discussed above are intended to be illustrative and notlimiting. One skilled in the art would appreciate that the steps of theprocesses discussed herein may be omitted, modified, combined, and/orrearranged, and any additional steps may be performed without departingfrom the scope of the invention. More generally, the above disclosure ismeant to be illustrative and not limiting. Only the claims that followare meant to set bounds as to what the present invention includes.Furthermore, it should be noted that the features and limitationsdescribed in any one embodiment may be applied to any other embodimentherein, and flowcharts or examples relating to one embodiment may becombined with any other embodiment in a suitable manner, done indifferent orders, or done in parallel. In addition, the systems andmethods described herein may be performed in real time. It should alsobe noted, the systems and/or methods described above may be applied to,or used in accordance with, other systems and/or methods. For example,the process of FIG. 10 can be performed on any of the devices shown inFIGS. 13-14 . Additionally, any of the steps in processes describedherein can be performed in any order, can be omitted, and/or can becombined with any of the steps from any other process.

1.-115. (canceled)
 116. A method comprising: storing a first media assetand metadata for the first media asset; causing displaying the firstmedia asset and text comments for the first media asset received fromother devices to which the first media asset was displayed; determiningthat a text string in the text comments is different from the metadatafor the first media asset; in response to the determining that the textstring in the text comments is different from the metadata for the firstmedia asset, identifying supplemental content based on the text string;and modifying the displaying of the first media asset to comprisedisplaying of the supplemental content.
 117. The method of claim 116,further comprising: identifying user input interacting with a particularcomment that comprises the text string; performing the identifyingsupplemental content based on the text string at least partially inresponse to identifying the user input interacting with the particularcomment.
 118. The method of claim 117, wherein the input interactingwith the particular comment comprises one or more of inputting a replyto the particular comment or selecting an interface elementcorresponding to a reaction to the particular comment.
 119. The methodof claim 117, wherein the input interacting with the particular commentcomprises hovering a cursor within a threshold distance of theparticular comment for greater than a threshold period of time.
 120. Themethod of claim 117, wherein the input interacting with the particularcomment comprises displaying the particular comment for greater than athreshold period of time.
 121. The method of claim 117, wherein theinput interacting with the particular comment comprises a command tocopy a portion of the particular comment.
 122. The method of claim 116,further comprising: determining that the text string corresponds to anitem; and analyzing the first media asset to identify the item at atimestamp in the first media asset; wherein the modifying the display ofthe media asset to comprise displaying of the supplemental contentfurther comprises causing display of the supplemental content at thetimestamp.
 123. The method of claim 122, further comprising: determininga location of the item in the first media asset; and detecting userinput interacting with the display of the media asset at the locationand timestamp and, in response, performing the modifying the display ofthe first media asset to comprise displaying the supplemental content.124. The method of claim 116, wherein the supplemental content is aninteractable element, and wherein the modifying the displaying of thefirst media asset to comprise displaying the interactable elementfurther comprises causing a portion of the first media asset to beinteractable via the interactable element.
 125. The method of claim 124,wherein the interactable element is a recommendation to view a secondmedia asset, and wherein an interaction with the interactable elementcauses displaying of the second media asset.
 126. A system comprising: amemory configured to store a first media asset and metadata for thefirst media asset; input/output circuitry configured to cause displayingthe first media asset and text comments for the first media assetreceived from other devices to which the first media asset wasdisplayed; and control circuitry configured to: determine that a textstring in the text comments is different from the metadata for the firstmedia asset; in response to the determining that the text string in thetext comments is different from the metadata for the first media asset,identify supplemental content based on the text string; and modify thedisplaying of the first media asset to comprise displaying of thesupplemental content.
 127. The system of claim 126, wherein the controlcircuitry is further configured to: identifying user input interactingwith a particular comment that comprises the text string; and performingthe identifying supplemental content based on the text string at leastpartially in response to identifying the user input interacting with theparticular comment.
 128. The system of claim 127, wherein the inputinteracting with the particular comment comprises one or more ofinputting a reply to the particular comment or selecting an interfaceelement corresponding to a reaction to the particular comment.
 129. Thesystem of claim 127, wherein the input interacting with the particularcomment comprises hovering a cursor within a threshold distance of theparticular comment for greater than a threshold period of time.
 130. Thesystem of claim 127, wherein the input interacting with the particularcomment comprises displaying the particular comment for greater than athreshold period of time.
 131. The system of claim 127, wherein theinput interacting with the particular comment comprises a command tocopy a portion of the particular comment.
 132. The system of claim 126,wherein the control circuitry is further configured to: determine thatthe text string corresponds to an item; and analyze the first mediaasset to identify the item at a timestamp in the first media asset;wherein when modifying the display of the media asset to comprisedisplaying of the supplemental content the control circuitry is furtherconfigured to cause display of the supplemental content at thetimestamp.
 133. The system of claim 132, wherein the control circuitryis further configured to: determine a location of the item in the firstmedia asset; and detect user input interacting with the display of themedia asset at the location and timestamp and, in response, perform themodifying the display of the first media asset to comprise displayingthe supplemental content.
 134. The system of claim 126, wherein thesupplemental content is an interactable element and wherein whenmodifying the displaying of the first media asset to comprise displayingthe interactable element the control circuitry is further configured tocause a portion of the first media asset to be interactable via theinteractable element.
 135. The method of claim 124, wherein theinteractable element is a recommendation to view a second media assetand wherein an interaction with the interactable element causesdisplaying of the second media asset. 136.-165. (canceled)